Real-Time End-to-End Action Detection with Two-Stream Networks
نویسندگان
چکیده
Two-stream networks have been very successful for solving the problem of action detection. However, prior work using two-stream networks train both streams separately, which prevents the network from exploiting regularities between the two streams. Moreover, unlike the visual stream, the dominant forms of optical flow computation typically do not maximally exploit GPU parallelism. We present a real-time end-to-end trainable two-stream network for action detection. First, we integrate the optical flow computation in our framework by using Flownet2. Second, we apply early fusion for the two streams and train the whole pipeline jointly end-to-end. Finally, for better network initialization, we transfer from the task of action recognition to action detection by pre-training our framework using the recently released large-scale Kinetics dataset. Our experimental results show that training the pipeline jointly end-to-end with fine-tuning the optical flow for the objective of action detection improves detection performance significantly. Additionally, we observe an improvement when initializing with parameters pre-trained using Kinetics. Last, we show that by integrating the optical flow computation, our framework is more efficient, running at real-time speeds (up to 31 fps). Keywords-Action detection, Two-stream networks, Video understanding
منابع مشابه
A Fuzzy Based Three Color Meter/Marker for Diffserv Networks (RESEARCH NOTE)
Differentiated Services (Diffserv) which was proposed by Internet Engineering Task Force (IETF), is a scalable and robust model for providing the end-to-end QoS. In the Diffserv networks, metering mechanisms are used to measure traffic stream. The single rate Three Color Meter (srTCM) [1],which was proposed by IETF, meters an IP packet stream and marks its packets either green, yellow, or red....
متن کاملExperimental Evaluation of Behavior-Based Failure-Detection Schemes in Real-Time Communication Networks
EEective detection of failures is essential for reliable communication services. Traditionally , non-real-time computer networks have relied on behavior-based techniques for detecting communication failures. That is, each node uses heartbeats to detect the failure of its neighbors , and the end-to-end transport protocol (e.g., TCP) achieves reliable communication by acknowledgment/retransmissio...
متن کاملOverload Management in Data Stream Processing Systems with Latency Guarantees
Stream processing systems are becoming increasingly important to analyse real-time data generated by modern applications such as online social networks. Their main characteristic is to produce a continuous stream of fresh results as new data are being generated at real-time. Resource provisioning of stream processing systems is difficult due to time-varying workload data that induce unknown res...
متن کاملMPEG-compliant joint sourceÕchannel coding using discrete cosine transform and substream scheduling for visual communication over packet networks
Quality of Service (QoS)-guarantee in real-time communication for multimedia applications is significantly important. An architectural framework for multimedia networks based on substreams or flows is effectively exploited for combining source and channel coding for multimedia data. But the existing frame by frame approach which includes Moving Pictures Expert Group (MPEG) cannot be neglected b...
متن کاملA Router - Based Congestion Control Scheme For Real - Time Continuous Media ( Dissertation Proposal )
This dissertation focuses on supporting the real-time transmission of continuous media over heterogeneous networks. Resource reservation is an effective solution when a fully QOS-capable connection exists from sender to receiver but many networks can not support reservations. End-to-end adaptations offer good best-effort attempts on networks where reservations are not possible but either degrad...
متن کاملذخیره در منابع من
با ذخیره ی این منبع در منابع من، دسترسی به آن را برای استفاده های بعدی آسان تر کنید
برای دانلود متن کامل این مقاله و بیش از 32 میلیون مقاله دیگر ابتدا ثبت نام کنید
ثبت ناماگر عضو سایت هستید لطفا وارد حساب کاربری خود شوید
ورودعنوان ژورنال:
- CoRR
دوره abs/1802.08362 شماره
صفحات -
تاریخ انتشار 2018